Abstract：Ordinal regression is a special machine learning paradigm and its objective is to classify patterns by using a between-class natural order property between the labels. Although many algorithms are proposed, the classical least squares regression (LSR) is not applied to the ordinal regression scenario. In this paper, a discriminative least squares ordinal regression (DLSOR) is proposed by using the cumulative labels and the margin-enlarging technique.Without constraints imposed on the regression function, DLSOR can embed ordinal information and expand between-class margin only through the label transformation. Thus, a high classification accuracy and low mean absolute errors can be guaranteed with the premise that the model complexity of DLSOR is consistent with that of LSR. The experimental results demonstrate the superiority of the proposed method in improving the ordinal regression performance.
[1] Herbrich R, Graepel T, Obermayer K. Large Margin Rank Boundaries for Ordinal Regression // Bartiett P J, Schlkopf B, Schuurmans D, et al., eds. Advances in Large Margin Classifiers. Cambridge, USA: MIT Press, 2000: 115-132 [2] Frank E, Hall M. A Simple Approach to Ordinal Classification // Proc of the 12th European Conference on Machine Learning. Freiburg, Germany, 2001: 145-156 [3] Waegeman W, Boullart L. An Ensemble of Weighted Support Vector Machines for Ordinal Regression. International Journal of Computer Systems Science and Engineering, 2009, 3(1): 47-51 [4] Chu W, Keerthi S S. New Approaches to Support Vector Ordinal Regression // Proc of the 22nd International Conference on Machine Learning. Bonn, Germany, 2005: 145-152 [5] Kramer S, Widmer G, Pfahringer B, et al. Prediction of Ordinal Classes Using Regression Trees // Proc of the 12th International Symposium on Methodologies for Intelligent Systems. Charlotte, USA, 2000: 426-434 [6] Cardoso J S, da Costa J F P. Learning to Classify Ordinal Data: The Data Replication Method. Journal of Machine Learning Research, 2007, 8: 1393-1429 [7] Lin H T, Li L. Reduction from Cost-Sensitive Ordinal Ranking to Weighted Binary Classification. Neural Computation, 2012, 24(5): 1329-1367
[8] Shashua A, Levin A. Ranking with Large Margin Principle: Two Approaches // Becker S, Thrun S, Obermayer K, eds. Advances in Neural Information Processing Systems. Cambridge, USA: MIT Press, 2002: 937-944 [9] Sun B Y, Li J Y, Wu D D, et al. Kernel Discriminant Learning for Ordinal Regression. IEEE Trans on Knowledge and Data Enginee-ring, 2010, 22(6): 906-910 [10] Liu Y, Liu Y, Chan K C C. Ordinal Regression via Manifold Learning // Proc of the 25th AAAI Conference on Artificial Intelligence. San Francisco, USA, 2011: 398-403 [11] Seah C W, Tsang I W, Ong Y S. Transductive Ordinal Regression. IEEE Trans on Neural Networks and Learning Systems, 2012, 23(7): 1074-1086 [12] Peerbhay K Y, Mutanga O, Ismail R. Commercial Tree Species Discrimination Using Airborne AISA Eagle Hyperspectral Imagery and Partial Least Squares Discriminant Analysis (PLS-DA) in Kwa Zulu-Natal, South Africa. ISPRS Journal of Photogrammetry and Remote Sensing, 2013, 79: 19-28 [13] Bakry A, Elgammal A. MKPLS: Manifold Kernel Partial Least Squares for Lipreading and Speaker Identification // Proc of the IEEE International Conference on Computer Vision and Pattern Recognition. Portland, USA, 2013: 684-691 [14] Xu Y L, Chen D R, Li H X. Least Square Regularized Regression for Multitask Learning [EB/OL].[2014-02-01]. http://www.hindawi.com/journals/aas/2013/715275 [15] Chen K, Gong S G, Xiang T, et al. Cumulative Attribute Space for Age and Crowd Density Estimation // Proc of the 26th IEEE International Conference on Computer Vision and Pattern Recognition. Portland, USA, 2013: 2467-2474 [16] Xiang S M, Nie F P, Meng G F, et al. Discriminative Least Squares Regression for Multiclass Classification and Feature Selection. IEEE Trans on Neural Networks and Learning Systems, 2012, 23(11): 1738-1754 [17] Wu Q, Ying Y M, Zhou D X. Learning Rates of Least-Square Regularized Regression. Foundations of Computational Mathema-tics, 2006, 6(2): 171-192 [18] Chen L, Tsang I W, Xu D. Laplacian Embedded Regression for Scalable Manifold Regularization. IEEE Trans on Neural Networks and Learning Systems, 2012, 23(6): 902-915 [19] Vapnik V N. The Nature of Statistical Learning Theory. 2nd Edition. NewYork, USA: Springer-Verlag, 2000 [20] Chang K Y, Chen C S, Hung Y P. Ordinal Hyperplanes Ranker with Cost Sensitivities for Age Estimation // Proc of the IEEE Conference on Computer Vision and Pattern Recognition. Providence, USA, 2011: 585-592 [21] Zhang Y, Yeung D Y. Multi-task Warped Gaussian Process for Personalized Age Estimation // Proc of the IEEE International Conference on Computer Vision and Pattern Recognition. San Francisco, USA, 2010: 2622-2629 [22] Chang C C, Lin C J. LIBSVM: A Library for Support Vector Machines[EB/OL].[2014-02-10]. http://www.csie.ntu.edu.tw/~cjlin/papers/libsvm.pdf
2015, Vol. 28 
